Device-free tracking system that accurately tracks hand movement

ABSTRACT

A method, system and computer program product for tracking movement of an object, such as a hand. Speakers of a device to be controlled transmit frequency modulated continuous wave (FMCW) audio signals. These signals are reflected by the object and received by the microphones at the controlled device. The received and transmitted audio signals are mixed. A fast Fourier transform (FFT) is then performed on the mixed audio signals. One or more peak frequencies in the frequency domain of the FFT mixed audio signals are selected and used to estimate the distance between the object and the speakers of the controlled device. Furthermore, the velocity of the object is estimated. The coordinates of the object are then computed using the estimated distance between the object and the speakers and microphones of the controlled device and the estimated velocity of the object.

TECHNICAL FIELD

The present invention relates generally to controlling devices, and moreparticularly to a device-free tracking system that accurately trackshand movement thereby enabling the control of the device by handmovement.

BACKGROUND

Smart TVs, smart appliances, Virtual Reality (VR), and Augmented Reality(AR) are all becoming increasingly popular. The key to their success ishaving an easy-to-use user interface to control the device (e.g., smartTVs, smart appliances and devices implementing VR/AR). Currently thoughsuch devices lack an easy-to-use user interface.

Smart TVs are currently cumbersome to control by having the usernavigate through various menus. Many smart appliances require users tomanually launch smartphone applications and click through pages tocontrol the smart appliance, which is even more cumbersome than turningon/off switches. VR and AR provide an immersive experience, and open thedoors to new ways of training, education, meeting, advertising, travel,health care, emergency responses, and scientific experiments. However,the current user interface of devices implementing VR/AR are ratherlimited: they rely on tapping, swiping, voice recognition, or steeringthe camera towards the hand to make sure the hand is within the view andline-of-sight of the camera while wearing the headset.

Hence, there is not currently a means for easily interacting andcontrolling devices, such as smart devices and device implementingVR/AR.

SUMMARY

In one embodiment of the present invention, a method for trackingmovement of an object comprises transmitting audio signals from one ormore speakers to the object. The method further comprises receivingsamples of the audio signals reflected from the object over a period oftime. The method additionally comprises mixing the received audiosignals with the transmitted audio signals. Furthermore, the methodcomprises performing, by a processor, a fast Fourier transform on themixed audio signals. Additionally, the method comprises selecting, bythe processor, one or more peak frequencies in a frequency domain of thefast Fourier transformed mixed audio signals. In addition, the methodcomprises estimating a velocity of the object. The method furthercomprises estimating, by the processor, a distance from a speaker of thecontrolled device to a microphone of the controlled device via theobject based on the selected one or more peak frequencies and velocityof the object.

Other forms of the embodiment of the method described above are in adevice and in a computer program product.

The foregoing has outlined rather generally the features and technicaladvantages of one or more embodiments of the present invention in orderthat the detailed description of the present invention that follows maybe better understood. Additional features and advantages of the presentinvention will be described hereinafter which may form the subject ofthe claims of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when thefollowing detailed description is considered in conjunction with thefollowing drawings, in which:

FIG. 1 illustrates a system configured in accordance with an embodimentof the present invention;

FIG. 2 illustrates a hardware configuration of a device to be controlledby the movement of a hand in accordance with an embodiment of thepresent invention;

FIG. 3 illustrates the chirp signal using Frequency Modulated ContinuousWave (FMCW) in accordance with an embodiment of the present invention;

FIG. 4 is a flowchart of a method for controlling a device using handmovements by allowing the controlled device to continuously track themovement of the hand using acoustic signals in accordance with anembodiment of the present invention;

FIG. 5 is a flowchart of the sub-steps of estimating the distancebetween the speakers of the controlled device in accordance with anembodiment of the present invention;

FIG. 6 shows a snapshot of the cross-correlation result in accordancewith an embodiment of the present invention;

FIG. 7 is a flowchart of the sub-steps of estimating the initialposition of the object, such as a hand, in accordance with an embodimentof the present invention;

FIG. 8A shows the simulation result of FMCW range detection when thetransmitter and receiver are both static and 0.5 m apart in accordancewith an embodiment of the present invention;

FIG. 8B illustrates the simulation result when the range is 0.5 m andthe Doppler shift is 20 Hz in accordance with an embodiment of thepresent invention;

FIGS. 9A-9B show the FMCW signals with and without the peaks in thespectral points, respectively, while the hand is moving in accordancewith an embodiment of the present invention;

FIG. 10 shows a snapshot of the received FMCW signal while the hand ismoving towards the microphone in accordance with an embodiment of thepresent invention; and

FIG. 11 shows an example of the Doppler shift estimation in accordancewith an embodiment of the present invention.

DETAILED DESCRIPTION

While the following discusses the present invention in connection withcontrolling a device (e.g., computers, game consoles, VR/AR headsets andsmart devices, such as smartphones and smart watches) using the movementof a hand, the principles of the present invention may be applied tocontrolling the device using the movement of any object (e.g., pen,pencil). A person of ordinary skill in the art would be capable ofapplying the principles of the present invention to suchimplementations. Further, embodiments applying the principles of thepresent invention to such implementations would fall within the scope ofthe present invention.

As discussed below, the present invention provides a device-free motiontracking system that enables a new way for users to interact with theworld by simply moving their hands. They can freely play video games,interact with VR/AR devices, and control smart appliances anywhere atany time. In one embodiment, such a tracking system uses widelyavailable speakers and microphones on the controlled device (e.g.,computers, game consoles, VR/AR headsets and smart devices, such assmartphones and smart watches). In one embodiment, the tracking systemutilizes a novel approach that estimates the distance to the speakerfrom the hand and the velocity of the hand using a single chirp signal.Such information may be used to accurately locate the moving hand. Byaccurately locating the moving hand, a user will be able to interact andcontrol devices by simply moving their hands.

As discussed below, in one embodiment, the device-free tracking systemof the present invention is based on Frequency Modulated Continuous Wave(FMCW). The device to be controlled includes one or more speakers andone or more microphones which are used to emit and receive audio signalsand track the hand movement. A pair of speakers and microphones may beco-located, and serve as an anchor point. Each speaker transmits chirpsignals in an inaudible and non-overlapping spectrum band with a guardband in between, and the microphone collects the signals from thecorresponding spectrum, mixes it with the transmitted signal, uses thepeak frequencies to estimate the distance and velocity, which are inturn used to track the hand movement.

As discussed further below, the process begins by detecting the start ofa “chirp” signal using cross correlation. Since the chirp signal isperiodic, it only needs to be detected once and audio samples can befetched continuously from the next sampling intervals (e.g., 100 ms).The distance between the speakers and the distance from the controlleddevice to the initial hand's position are estimated. Then audio signalsare continuously fetched. Fast Fourier Transform (FFT) is performed onthe fetched audio signals to detect the peak frequencies. It has beenobserved that the mixed signal in FMCW has a fundamental frequencydetermined by the parameters of the chirp sequence. This property isleveraged to filter out the reflection from the static objects anddetect the reflection caused by the moving hand. Next, the Doppler shiftis estimated and used to select the appropriate FMCW peak for distanceestimation (distance between the controlled device and the hand). Thisdistance estimation as well as the velocity estimation of the hand(obtained from the Doppler shift) are then used to continuously trackthe hand. A more detailed explanation of this process is provided below.

In the following description, numerous specific details are set forth toprovide a thorough understanding of the present invention. However, itwill be apparent to those skilled in the art that the present inventionmay be practiced without such specific details. For the most part,details considering timing considerations and the like have been omittedinasmuch as such details are not necessary to obtain a completeunderstanding of the present invention and are within the skills ofpersons of ordinary skill in the relevant art.

Referring now to the Figures in detail, FIG. 1 illustrates a system 100configured in accordance with an embodiment of the present invention.Referring to FIG. 1, system 100 includes a device to be controlled 101(referred to herein as “controlled device”) by a hand 102 of a user.Controlled device 101 may be any computing device that contains speakers103A-103B (identified as “speaker A” and “speaker B,” respectively, inFIG. 1, respectively) and two microphones 104A-104B (identified as“microphone A” and “microphone B,” respectively, in FIG. 1). Speakers103A-103B may collectively or individually be referred to as speakers103 or speaker 103, respectively. Microphones 104A-104B may collectivelyor individually be referred to as microphones 104 or microphone 104,respectively. While FIG. 1 illustrates controlled device 101 asincluding two speakers 103 and two microphones 104, controlled device101 of the present invention is not to be limited to only including twospeakers 103 and two microphones 104. Instead, controlled device 101 mayinclude one or more speakers 103 and one or more microphones 104. Someexamples of controlled devices 101 include but not limited to computers,video game consoles, VR/AR headsets, smartphones, smart watches,wearable devices, smart TVs and smart appliances. Controlled device 101is configured to emit an audio signal through its speakers 103, whetheraudible or inaudible to humans. Furthermore, controlled device 101 isconfigured to receive an audio signal that is reflected by the user'shand 102 by microphones 104. This reflected audio signal is used bycontrolled device 101 to continuously track hand 102 in real time asdiscussed further below. A more detailed description of a hardwareconfiguration of an embodiment of controlled device 101 is providedbelow in connection with FIG. 2.

Referring now to FIG. 2, FIG. 2 is a functional block diagram of anexample of a controlled device 101 (FIG. 1). In this example, controlleddevice 101 includes one or more processors 201. Processor 201 caninclude one or more application specific integrated circuits (ASICs),digital signal processors (DSPs), digital signal processing devices(DSPDs), programmable logic devices (PLDs), field programmable gatearrays (FPGAs), controllers, micro-controllers, microprocessors,electronic devices, electronic units, or a combination thereof

Processor 201 is configured to store data received by one or moreinterfaces and process and store the data on a memory 202. Memory 202can be implemented within processor 201 or external to processor 201. Asused herein, the term memory refers to any type of long term, shortterm, volatile, nonvolatile, or other memory and is not to be limited toany particular type of memory or number of memories or type of mediaupon which memory is stored. In one embodiment, memory 202 stores anapplication, such as a program for controlling device 101 using handmovements by hand 102 (FIG. 1) by allowing the controlled device 101 tocontinuously track the movement of hand 102 using acoustic signals. Inone embodiment, processor 201 is configured to execute the programinstructions of applications stored in memory 202.

Furthermore, speakers 103A, 103B and microphones 104A, 104B areconnected to controlled device 101 via a user interface adapter 203.Speakers 103A, 103B are configured to generate an audio signal (audibleor inaudible to humans) at various frequencies. Furthermore, microphones104A, 104B are configured to receive an audio signal that is reflectedby the user's hand 102.

Controlled device 101 of FIG. 2 is not to be limited in scope to theelements depicted in FIG. 2 and may include fewer or additional elementsthan depicted in FIG. 2.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

As stated in the Background section, smart TVs, smart appliances,Virtual Reality (VR), and Augmented Reality (AR) are all becomingincreasingly popular. The key to their success is having an easy-to-useuser interface to control the device (e.g., smart TVs, smart appliancesand devices implementing VR/AR). Currently though such devices lack aneasy-to-use user interface. Smart TVs are currently cumbersome tocontrol by having the user navigate through various menus. Many smartappliances require users to manually launch smartphone applications andclick through pages to control the smart appliance, which is even morecumbersome than turning on/off switches. VR and AR provide an immersiveexperience, and open the doors to new ways of training, education,meeting, advertising, travel, health care, emergency responses, andscientific experiments. However, the current user interface of devicesimplementing VR/AR are rather limited: they rely on tapping, swiping,voice recognition, or steering the camera towards the hand to make surethe hand is within the view and line-of-sight of the camera whilewearing the headset. Hence, there is not currently a means for easilyinteracting and controlling devices, such as smart devices and deviceimplementing VR/AR.

The principles of the present invention provide a means for easilyinteracting and controlling devices, such as smart devices and devicesimplementing VR/AR, by continuously tracking the movement of a hand(e.g., hand 102 of FIG. 1) of a user as discussed below in associationwith FIGS. 3-7, 8A-8B, 9A-9B and 10-11. FIG. 3 illustrates the chirpsignal using Frequency Modulated Continuous Wave (FMCW). FIG. 4 is aflowchart of a method for controlling a device using hand movements byallowing the controlled device to continuously track the movement of thehand using acoustic signals. FIG. 5 is a flowchart of the sub-steps ofestimating the distance between the speakers of the controlled device.FIG. 6 shows a snapshot of the cross-correlation result. FIG. 7 is aflowchart of the sub-steps of estimating the initial position of theobject, such as a hand. FIG. 8A shows the simulation result of FMCWrange detection when the transmitter and receiver are both static and0.5 m apart. FIG. 8B illustrates the simulation result when the range is0.5 m and the Doppler shift is 20 Hz. FIGS. 9A-9B show the FMCW signalswith and without the peaks in the spectral points, respectively, whilethe hand (e.g., hand 102 of FIG. 1) is moving. FIG. 10 shows a snapshotof the received FMCW signal while the hand is moving towards themicrophone. FIG. 11 shows an example of the Doppler shift estimation.

A brief discussion regarding Frequency Modulated Continuous Wave (FMCW)radar which is a widely used technique in radar to measure the distanceto an object is deemed appropriate.

One way to estimate the distance d is to directly measure thepropagation delay τ and use their relationship d=τ*υ, where υ is thepropagation speed of the signal. However, this is challenging because alarge bandwidth is needed in order to send a sharp pulse signal with agood time resolution. Instead, FMCW indirectly estimates the propagationdelay based on the frequency shift of the chirp signal as follows asshown in FIG. 3. The “chirp” signal as used herein, refers to a signalin which the frequency increases (up-chirp) or decreases (down-chirp)with time.

FIG. 3 illustrates the chirp signal using FMCW in accordance with anembodiment of the present invention. As illustrated in FIG. 3, curve 301shows a transmission chirp, whose frequency linearly increases overtime. Curve 302 shows the received chirp. Let f_(c), B, and T denote thecarrier frequency, bandwidth, and duration of the chirp, respectively.The frequency of the signal at time t is given by

${f(t)} = {f_{c} + {\frac{Bt}{T}.}}$

The phase of the signal is calculated by integrating f(t) over time,which is:

${u(t)} = {2{{\pi\left( {{f_{c}t} + {B\frac{t^{2}}{2T}}} \right)}.}}$

As a result, the transmitted chirp signal can be represented asυ_(tx)(t)=cos(u(t)), where its magnitude is assumed to be one forsimplicity.

Consider that the transmitter and receiver are co-located. Thetransmitted signal is reflected by the target and received with thedelay τ_(d). Therefore, the received signal at time t isυ_(rx)(t)=υ_(tx)(t−τ_(d)), which is:

${{\upsilon_{rx}(t)} = {\cos \left( {{2\pi \; {f_{c}\left( {t - \tau_{d}} \right)}} + \frac{\pi \; {B\left( {t - \tau_{d}} \right)}^{2}}{T}} \right)}},$

where again the magnitude change is ignored. Let R, V_(c), V denote thedistance between the transceiver and target, the propagation speed, andthe target's velocity. Then, the reflected signal is delayed by τ_(d):

$\begin{matrix}{\tau_{d} = {\frac{2\left( {R + {Vt}} \right)}{V_{c}}.}} & (1)\end{matrix}$

The receiver multiplies the transmitted signal with the received signalin the time domain, which is υ_(m)(t)=υ_(tx)(t) υ_(rx)(t). The mixedsignal υ_(m)(t) is called the Intermediate Frequency (IF) signal. Byusing cos A cos B=(cos(A−B)+cos(A+B))/2 and filtering out the highfrequency cos(A+B) component, which has the frequency of the order2f_(c), υ_(m)(t) becomes:

${\upsilon_{m}(t)} = {{\cos \left( {{2\pi \; f_{c}\tau_{d}} + {2\pi \; t\frac{{B\; \tau_{d}}\;}{T}} + \frac{2\pi \; B\; \tau_{d}^{2}}{T}} \right)}.}$

By plugging Equation (1) into the above equation, υ_(m)(t) becomes:

$\begin{matrix}{{\upsilon_{m}(t)} = {{\cos \left( {{4\pi \; f_{c}\frac{R}{V_{c}}} + {2{\pi \left( {\frac{2f_{c}V}{V_{c}} + \frac{2{RB}}{V_{c}T}} \right)}t} + \frac{2{\pi \left( {R + {Vt}} \right)}^{2}}{{Tc}^{2}}} \right)}.}} & (2)\end{matrix}$

The constant phase terms and quadratic term are ignored since the formerdoes not change the frequency and the latter is too small to matter.Furthermore, the frequency of the IF signal f_(jF) is approximated as:

$\begin{matrix}{f_{IF} = {f_{R} + f_{V}}} & (3) \\{f_{R} = \frac{2{RB}}{V_{c}T}} & (4) \\{f_{V} = \frac{2f_{c}V}{V_{c}}} & (5)\end{matrix}$

As can be seen, the frequency shift includes (i) the frequency shiftthat is proportional to the distance to the target f_(R), and (ii) theDoppler shift f_(V) due to the movement of the target. The formerdepends on the distance and the latter depends on the velocity. Theseeffects also are shown in FIG. 3, which compares the frequency changewith and without movement and shows an additional shift coming from theDoppler shift when the target moves.

Turning now to FIG. 4, FIG. 4 is a flowchart of a method 400 forcontrolling a device (e.g., device 101 of FIGS. 1 and 2) using handmovements by allowing controlled device 101 to continuously track themovement of an object, such as a hand (e.g., hand 102 of FIG. 1) usingacoustic signals in accordance with an embodiment of the presentinvention.

Referring to FIG. 4, in conjunction with FIGS. 1-3, in step 401,controlled device 101 estimates the distance between speaker 103 andmicrophone 104 of controlled device 101. A further description of thesteps involved in estimating the distance between speakers 103 ofcontrolled device 101 is provided below in connection with FIG. 5.

One can measure the distance between speakers 103 using a ruler or usinga smartphone based calibration. Below introduces an even simplerprocedure without any user intervention if microphones 104 areco-located with speakers 103.

FIG. 5 is a flowchart of the sub-steps of estimating the distancebetween speakers 103 of controlled device 101 (step 401) in accordancewith an embodiment of the present invention.

Referring to FIG. 5, in conjunction with FIGS. 1-4, in step 501,controlled device 101 transmits Frequency Modulated Continuous Waves(FMCW) audio signals (whether audible or inaudible to humans), includinga first chirp signal, from speaker(s) 103 to an object, such as hand102.

In step 502, microphone 104 of controlled device 101 receives thesamples of audio signals reflected from the object (e.g., hand 102) overa period of time.

In step 503, controlled device 101 detects the transmission of a firstchirp signal from speaker 103 using cross correlation between thereceived and transmitted audio signals.

In step 504, controlled device 101 mixes the received audio signal withthe transmitted audio signals.

In order to mix the transmitted and the received signal (transmittedsignal refers to the signal transmitted by speaker 103 and the receivedsignal refers to the transmitted signal reflected from the user's hand102), the program of the present invention should know the exact starttime of the chirp signal transmission. This is challenging even if theaudio signal transmission and the reception are implemented in the sameprogram and running on the same machine, due to a random delay betweenthe call of the audio play function and the play time of the audio file.To avoid the uncertainty, the beginning of the chirp signal is detectedusing cross-correlation between the received audio signal and theoriginal chirp signal. As speaker 103 and microphone 104 are co-located,the signal is considered to be received as soon as it is transmitted. Byfinding the peak in the cross-correlation output, the start of the chirpsequence in the received signal can be detected, which is also the startof the transmitted chirp signal. FIG. 6 shows one snapshot of thecross-correlation result in accordance with an embodiment of the presentinvention. In an illustrative embodiment, FIG. 6 shows the peak every480 samples, which corresponds to an exemplary chirp duration.

In step 505, controlled device 101 performs fast Fourier Transform (FFT)on the mixed audio signals. In one embodiment, controlled device 101further filters out the reflection from the static objects.

In step 506, controlled device 101 selects a peak frequency in thefrequency domain of the FFT mixed audio signals.

In step 507, controlled device 101 estimates the distance betweenspeaker 103 and microphone 104 of controlled device 101 using theselected peak frequency.

Suppose the transmitted and received signals are synchronized using theabove procedure. One can measure the distance from the left speaker(e.g., speaker 103 A) to the right microphone (e.g., microphone 104B)using the FMCW signal transmitted from the left speaker based onEquation (4). Likewise, one can measure the same distance from the rightspeaker (e.g., speaker 103B) to the left microphone (e.g., microphone104A). To improve the accuracy, one can average the distance measuredfrom both directions. One can further measure the distances for a longerduration and then obtain the average since speakers 103 are static.

In step 508, controlled device 101 estimates the distance betweenspeakers 103 of controlled device 101 using the estimated distancebetween speaker 103 and microphone 104 of controlled device 101.

In one embodiment, a speaker 103 and a microphone 104 are co-located andthe distance between speakers 103 of controlled device 101 is equal tothe distance between a speaker 103 and another speaker's 103 co-locatedmicrophone 104. In another embodiment, all speakers 103 and microphones104 are placed along one line, and the distance between any two speakers103 can be estimated based on the distances between the pairs of aspeaker 103 and a microphone 104 (e.g., estimating the distance betweenspeaker 1 and speaker 2 based on the distances between speaker 1 andmicrophone 1, speaker 2 and microphone 2, speaker 1 and microphone 2,and speaker 2 and microphone 1).

Returning now to FIG. 4, in conjunction with FIGS. 1-3, in step 402,controlled device 101 estimates the initial position of the object, suchas hand 102. A further description of the steps involved in estimatingthe initial position of the object, such as hand 102, is provided belowin connection with FIG. 7.

FIG. 7 is a flowchart of the sub-steps of estimating the initialposition of the object (step 402) in accordance with an embodiment ofthe present invention.

Referring to FIG. 7, in conjunction with FIGS. 1-6, in step 701,controlled device 101 transmits Frequency Modulated Continuous Waves(FMCW) audio signals (whether audible or inaudible to humans), includinga first chirp signal, from speaker(s) 103 to the object, such as hand102, as in step 501.

In step 702, microphone 104 of controlled device 101 receives thesamples of audio signals reflected from the object (e.g., hand 102 )over a period of time as in step 502.

In step 703, controlled device 101 detects the transmission of a firstchirp signal from speaker 103 using cross correlation between thereceived and transmitted audio signals as in step 503.

In step 704, controlled device 101 mixes the received audio signal withthe transmitted audio signals as in step 504.

In step 705, controlled device 101 performs fast Fourier Transform (FFT)on the mixed audio signals as in step 505.

In step 706, controlled device 101 detects the initial hand gestureusing pattern matching based on the frequency domain of the mixed audiosignals.

A user initializes tracking by performing an initial gesture. In oneembodiment, an initial gesture may correspond to a grabbing gesture(e.g., closing and opening a hand twice) as the initial gesture. In oneembodiment, the initial gesture can be recognized using patternmatching. When a user performs the initial gesture, the received signalreflected from the object (e.g., hand 102) is recorded. The receivedsignal is mixed with the transmitted signal, and fast Fourier transform(FFT) is performed on the mixed trace. The FFT of the mixed trace isrecorded. The above process may be repeated to obtain multiple positivetraining traces while the user is performing the initial gesture.Similarly, negative training traces from the FFT of the mixed traces arecollected while the user is not performing the initial gesture. Todetect the initial gesture later, recently received FFTs of the mixedtrace are matched against the positive or negative training trace usingpattern matching algorithms to detect an initial gesture.

In step 707, controlled device 101 filters out the reflection from thestatic objects. That is, spectral points are filtered resulting from thereflection from the static objects.

In step 708, controlled device 101 selects a peak frequency in thefrequency domain of the FFT mixed audio signals as in step 506.

In step 709, controlled device 101 estimates the distance betweenspeaker(s) 103 of controlled device 101 and the object, such as hand102, using the selected peak frequency.

In step 710, controlled device 101 estimates the initial position of theobject, such as hand 102, using the estimated distance betweenspeaker(s) 103 and microphone(s) 104 of controlled device 101 and theestimated total distance from speaker 103 of controlled device 101 tomicrophone 104 of controlled device 101 via the object, such as hand102.

The estimated distance is the sum of the distance from speaker 103 tothe object (e.g. hand 102) and the distance from the object tomicrophone 104. If the pair consisting of speaker 103 and microphone 104is co-located, the distance between speaker 103 and the object is halfthe estimated distance. When the number of speakers 103 and the numberof dimensions are both equal to 2, the position of hand 102 is estimatedas the intersection of all the circles centered

$R = {\frac{\Delta \; {fV}_{c}T}{2B}.}$

at each of the speakers 103 with the radius derived from Similarly, ifboth the number of speakers 103 and dimensions are 3, the solution isthe intersection of the spheres centered at these speakers 103 with thecorresponding radii. If there are more speakers 103 than the number ofdimensions (i.e., there are more constraints than the unknowns), one canlocalize hand 102 by solving the following simple optimization:

${\min \text{:}{\sum\limits_{i}^{\;}{{{{dist}\left( {P - A_{i}} \right)} - {dist}_{i}}}^{2}}},$

where P is the coordinate of hand 102, A, is the i-th speaker'scoordinate, and dist, is the distance between the i-th speaker and hand102 estimated using FMCW. Essentially, this objective minimizes thetotal error with respect to all the FMCW distance measurements fromspeakers 103.

If each pair of speaker 103 and microphone 104 is not co-located, theobject is located at the intersection of ellipses (2D) or ellipsoids(3D) whose foci are at speaker 103 and microphone 104 respectively, andthe total distance to the foci is the corresponding estimated distance.Therefore, the object's position can be estimated based on the totaldistance to the foci.

Returning to FIG. 4, in conjunction with FIGS. 1-3, in step 403,controlled device 101 transmits Frequency Modulated Continuous Waves(FMCW) audio signals (whether audible or inaudible to humans) fromspeaker(s) 103 to the object, such as hand 102.

In step 404, microphone 104 of controlled device 101 receives samples ofthe audio signals reflected from the object, such as hand 102, that wereemitted by speaker(s) 103 over a period of time.

In step 405, controlled device 101 mixes the received audio signals withthe transmitted audio signals.

In step 406, controlled device 101 performs fast Fourier Transform (FFT)on the mixed audio signals.

In step 407, controlled device 101 selects one or more peak frequenciesin the frequency domain of the FFT mixed audio signals.

When hand 102 is performing the initial gesture, FFT is performed todetermine the peak frequencies in the mixed signals. Then thefrequencies are filtered out at the spectral points to remove reflectionfrom the static objects. Next, one of the remaining peaks is selectedbased on its magnitude and/or consistency with the velocity estimatedfrom the actual Doppler shift. The selected peak is rounded to theclosest spectral point, and the rounded frequency is converted to thedistance estimate using Equation (4). This procedure works because theinitial gesture has slow speed, well below 50 Hz.

Consider a chirp signal (shown in FIG. 3) propagates over the medium andarrives at the receiver after a delay τ_(d). As shown in Equation (3),without movement the range (distance between speaker 103 and hand 102)can be estimated based on the frequency shift as follows:

$R = {\frac{I_{p}{cT}}{B}.}$

The transmitted chirp signal has fundamental frequencies that are allmultiples of the frequency 1/T. In other words, in the frequency domain,it has spectral points with an interval of 1/T Hz. For example, when thechirp interval is 0.01 s, it has spectral points every 100 Hz. Thereceived signal υ_(r)(t) is a simple time shifted version of thetransmitted signal υ_(t)(t), and has the same period.υ_(m)(t)=υ_(r)(t)υ_(t)(t), has the same period since the periodicity ispreserved when two periodic signals with the same period are multiplied.Therefore, the frequency shifts in FMCW exhibit peaks at discretefrequencies, all of which are multiples of 1/T Hz without any movement.

FIG. 8A shows the simulation result of FMCW range detection when thetransmitter and receiver are both static and 0.5 m apart in accordancewith an embodiment of the present invention. The carrier frequency,bandwidth, and the duration of the chirp signal are 17 KHz, 2 KHz, and0.01 s, respectively. FIG. 8A is the FFT result of υ_(m)(t). υ_(m)(t)has power every 100 Hz. There are two major peaks at 0 Hz and 600 Hz.The peak at 0 Hz is caused by the self-interference that the transmittedsignal is directly received without reflection. The self-interferencecan be filtered by ignoring the peak around 0 Hz. Then another peak at600 Hz is detected and considered to be due to the reflection from thetarget. Using Equation (4), one can determine that the total distance is1.02 m, and the distance from the transceiver to the target is half ofthat distance: 0.51 m since the total distance includes both forward andreflected paths.

Initial experiments have shown that the above FMCW approach yields roughestimation of the hand position but the error is significant: 10 cm,which is too large for many applications. There are several factors thatcontribute to the large error. First, there are multiple peaks, some ofwhich are due to reflection from other objects. Second, even ignoringthe reflection from static objects, the signals can still be reflectedby multiple body parts that move together with hand 102. These bodyparts have different distances from speakers 103. Moreover, evenconsidering reflection from the same point, the reflected signal maystill traverse different paths to reach microphone 104. These effectstogether contribute to multiple FMCW peaks. It is important to selectthe peak that comes through the direct path from hand 102. Third, thefrequency change in FMCW partly comes from the propagation delay andpartly comes from the Doppler shift. This is shown in FIG. 8B, whichplots the simulation result when the range is 0.5 m and the Dopplershift is 20 Hz in accordance with an embodiment of the presentinvention. In this case, a combined frequency shift at 620 Hz isobserved. It is important to decouple the overall shift to thedistance-based shift and Doppler shift to achieve high accuracy.

Reflected signals are challenging to use for tracking because there arepossibly many objects around the target that reflect the signal. Thereare multiple peaks in the FMCW. As a result, the peaks caused by thehand movement should be extracted. It is reasonable to assume only hand102 is moving and the other objects are static. Based on thisassumption, one can ignore all peaks at the spectral points (i.e.,integer multiples of the fundamental frequency) since they do not haveDoppler shift. FIGS. 9A-9B show the FMCW signals with and without thepeaks in the spectral points, respectively, while the hand is moving inaccordance with an embodiment of the present invention. Ignoring thespectral points, one can clearly observe the shift caused by the movinghand 102.

In step 408, controlled device 101 estimates the “pseudo Doppler shift”in the frequency domain of the FFT mixed audio signals for each selectedpeak frequency, where the pseudo Doppler shift is the difference betweenthe current peak frequency versus the closest spectral point below thecurrent peak frequency in the FFT mixed audio signals.

The notion of “pseudo Doppler shift” is now introduced. It is defined asthe difference between the current peak frequency versus the closestspectral point below the current peak frequency. If the fundamentalfrequency is 100 Hz and the peak frequency is 620 Hz, then the pseudoDoppler shift is 20 Hz. Similarly, a peak frequency at 660 Hz has thepseudo Doppler shift of 60 Hz. In this case, the actual Doppler shiftmay be 60 Hz or −40 Hz. As discussed further below, the pseudo Dopplershift is translated to the actual Doppler shift. It is noted that thepseudo Doppler shift is defined as the difference with respect to thespectral point below the current peak frequency for consistency. Forexample, suppose there are two peaks at 660 Hz and 540 Hz. To combinethe two estimates, one should combine 60 Hz with 40 Hz, instead of −40Hz with 40 Hz.

In step 409, controlled device 101 combines the multiple pseudo Dopplershifts into one pseudo Doppler shift.

As mentioned earlier, multiple peaks exist even after filtering out thepeaks caused by reflection from static objects because the signal isreflected by different body parts that move together with the hand andthey may have different distances from speaker 103. Interestingly, thereis a benefit of getting multiple peaks—if some or all of the peakscorrespond to the body parts with similar velocities, one essentiallyobtains multiple estimates of the Doppler shift and can combine thesemultiple estimates to enhance the accuracy using Maximum Ratio Combining(MRC). Specifically, one can use the highest peak and multiple peaksaround the highest peak to estimate the pseudo Doppler shift as follows.

1. One can divide the whole spectrum into multiple spectrum slots thatspan from

${\left\lbrack {\frac{n}{T} - \frac{1}{2T}} \right\rbrack \mspace{14mu} {Hz}\mspace{14mu} {{to}\left\lbrack {\frac{n}{T} + \frac{1}{2T}} \right\rbrack}\mspace{14mu} {Hz}},$

where n is an integer smaller than B×T. For example, when T is 0.01 s,the n-th slot spans (n×100−50, n×100+50).

2. The slot with the maximum peak is searched and denoted as k. Thepseudo Doppler shift is computed in the slot k and its 4 nearby slots.

3. The pseudo Doppler shifts are combined in these 5 slots using thefollowing weighted average:

${f_{D} = \frac{\sum\limits_{i = {k - 2}}^{i = {k + 2}}{\sigma_{i}f_{d,i}}}{\sum\limits_{i = {k - 2}}^{i = {k + 2}}\sigma_{i}}},$

where σ_(i) and f_(d,I) denote the peak magnitude and pseudo Dopplershift during the i-th slot, respectively. The highest peak and itsnearby peaks are used for the Doppler estimation of hand 102, becausehis/her hand 102 is the closest moving object to speakers 103 when auser is facing the controlled device 101 (a common usage scenario).

As an example, FIG. 10 shows a snapshot of the received FMCW signalwhile hand 102 is moving towards microphone 104 in accordance with anembodiment of the present invention. As illustrated in FIG. 10, thehighest peak is in slot 6. The peaks from slots 4 to 8 are combined toget the final estimate of the pseudo Doppler shift.

In step 410, controlled device 101 translates the pseudo Doppler shiftto the actual Doppler shift.

As mentioned above, the frequency shift comes partly from thepropagation delay and partly from the Doppler shift. To decompose thefrequency shift, it has been observed that the shift caused by thepropagation delay should be multiples of the fundamental frequency andthe difference between the spectral point and the frequency of thecurrent peak is caused by the Doppler shift. It has been found that thehand movement is typically within 1 m/s, and its corresponding frequencyshift should be within 100 Hz. Given the overall frequency change, thereare two choices: reminder(currFreq,FF) or 100-reminder(currFreq,FF). Inthe previous example, the Doppler shift in the 660 Hz frequencydifference can be either reminder(660,100)=60 Hz orreminder(660,100)−100=−40 Hz.

One approach is to select the Doppler shift with the smallest magnitude.For example, one selects −40 Hz instead of 60 Hz when the peak frequencyis at 660 Hz. However, this selection may be off since occasionally hand102 can generate more than a 50 Hz Doppler shift. FIG. 11 shows anexample of the Doppler shift estimation in accordance with an embodimentof the present invention. At 1.1 seconds, the actual Doppler shift is 59Hz, but this scheme (see curve 1101) estimates −41 Hz, which results ina significant tracking error.

To enhance the accuracy, a simple search is used based on the previousmovement (see curve 1102 “search”). It tries each of the two choicesevery slot and picks the combination that minimizes the velocity changeover all the slots. One can vary the window size (number of slots usedto compute the velocity change) to run the exhaustive search. Resultshave shown that a window size of 1, which essentially picks the currentDoppler shift that is closest to the previous Doppler shift, is alreadygood enough. This is because the Doppler shift of the hand movementduring the first 0.1-second slot (starting from no movement) is likelyto be small: well below 50 Hz and easy to select from the twocandidates. Once the Doppler shift of the previous slot is correct, thenext one will be correct in a greedy algorithm (assuming minimizing thevelocity change leads to the correct selection). In FIG. 11, the“search” (see curve 1102) selects the correct Doppler shift. Therefore,the MRC output discussed above is used to estimate the pseudo Dopplershift and the “search” is applied to translate the pseudo Doppler shiftto the actual Doppler shift.

In step 411, controlled device 101 estimates the velocity of themovement of the object, such as hand 102, using the actual Dopplershift. The velocity estimate of the movement of hand 102 is thenestimated using the estimated actual Doppler shift in Equation (5) asdiscussed above.

In step 412, controlled device 101 selects one or more peak frequenciesin the frequency domain of the FFT mixed signals based on the estimatedvelocity of the movement of the object.

Due to the multipath, the highest peak may not come through the directpath from hand 102. One can use the above Doppler estimate to refine therange estimation (“range estimation” refers to the distance between hand102 and controlled device 101, specifically speaker 103 of controlleddevice 101). In particular, instead of using the frequency of thehighest peak to estimate the range, a peak whose distance change is mostconsistent with the one estimated by the Doppler shift is found. This isbased on the observation that Doppler estimation in a short interval ismore reliable than the range estimation. Specifically, let a denote thevelocity from the current Doppler estimation and t_(s) denote thesampling interval. Then υ·t_(s) is the distance change during thecurrent interval estimated using the velocity. Meanwhile, one canestimate the distance change from the FMCW. For each of the five peaksnear the highest peak, the distance change from the previous position iscomputed (i.e., |d^(i)(t)−d(t−1)|)), where d(t−1) is the previousdistance and d_(i)(t) is the current distance derived using the i-thFMCW peak according to Equation (3). The particular i that is selectedis the one that minimizes ∥d^(u)(t), d(i−1)|−υ·t_(s)|. d^(i)(t) is thenused as the range in the current interval.

In step 413, controlled device 101 estimates the distance between theobject, such as hand 102, and speaker 103 of controlled device 101 basedon the selected one or more peak frequencies using Equation (4).

In step 414, controlled device 101 computes the coordinates of theobject, such as hand 102, using the estimated velocity of the object,such as hand 102, the estimated distance between speakers 103 andmicrophones 104 of controlled device, the estimated total distance fromspeaker 103 of controlled device 101 to microphone 104 of controlleddevice 101 via the object and the initial position of the object, suchas hand 102. Upon computing the coordinates of the object, such as hand102, controlled device 101 continues to receive samples of the audiosignals reflected from the object, such as hand 102, that were emittedby speaker(s) 103 over the next period of time (e.g., 100 ms) in step404.

Given the initial position of hand 102 discussed above, one cancontinuously track the hand's movement using the range and the Dopplershift together. One might wonder why one would not track the movement ofhand 102 solely based on the range. This is because using range alonedoes not provide a high enough accuracy and exploiting the Doppler shiftimproves the tracking accuracy. Let R_(k)[t] and f_(D,k)[t] denote thedistance and Doppler shift from the anchor point k at time t,respectively. In addition, the distance measurement is obtained usingthe velocity from the Doppler shift and the previous position (which isderived from the initial position). The Doppler based distanceR_(D,k)[t] is as follows:

${{R_{D,k}\lbrack t\rbrack} = {{{D_{k}\left\lbrack {t - 1} \right\rbrack} + {\upsilon \cdot t_{s}}} = {{D_{k}\left\lbrack {t - 1} \right\rbrack} + {\left( {\frac{f_{D,k}\lbrack t\rbrack}{f_{k}}V_{c}} \right)t_{s}}}}},$

where D_(k)[t−1] is the distance from the k-th speaker at the time slott−1, f_(k) is the carrier frequency of speaker 103, V_(c) is thepropagation speed of the audio signal, and t_(s) is the FMCW samplinginterval.

Using two independent distance measurements R_(k)[t] and R_(D,k)[t], thecurrent position (x, y) can be found by solving the followingoptimization problem:

${{\min\limits_{x,y}{\text{:}\alpha {\sum\limits_{k = 1}^{2}\left( {\sqrt{\left( {x_{k} - x} \right)^{2} + \left( {y_{k} - y} \right)^{2}} - {R_{k}\lbrack t\rbrack}} \right)^{2}}}} + {\beta {\sum\limits_{k = 1}^{2}\left( {\sqrt{\left( {x_{k} - x} \right)^{2} + \left( {y_{k} - y} \right)^{2}} - {R_{D,k}\lbrack t\rbrack}} \right)^{2}}}},$

where (x_(k), y_(k)) is the position of the k-th speaker, and α and βare the constant weighting factors determined by the reliability of therange and Doppler shift estimation results, respectively. In theexemplary evaluation, α was set to 0.5 and β was set to 1 since theDoppler shift was found to be more accurate than FMCW.

Hence, the device-free motion tracking system of the present inventionprovides a new way for users to interact with the world by simply movingtheir hands. Users can freely play video games, interface with VR/ARdevise, and control smart appliances anywhere at any time.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

1. A method for tracking movement of an object, the method comprising:transmitting audio signals from one or more speakers to said object;receiving samples of said audio signals reflected from said object overa period of time; mixing said received audio signals with saidtransmitted audio signals; performing, by a processor, a fast Fouriertransform on said mixed audio signals; selecting, by said processor, oneor more peak frequencies in a frequency domain of said fast Fouriertransformed mixed audio signals; estimating a velocity of said object;and estimating, by said processor, a distance from a speaker of saidcontrolled device to a microphone of said controlled device via saidobject based on said selected one or more peak frequencies and velocityof said object.
 2. The method as recited in claim 1 further comprising:estimating coordinates of said object using said estimated distance fromsaid speaker of said controlled device to said microphone of saidcontrolled device via said object, said estimated velocity of saidobject, estimated distances between all speakers of said controlleddevice and an estimated distance between each pair of said speakers andmicrophones of said controlled device.
 3. The method as recited in claim1 further comprising: estimating a pseudo Doppler shift in saidfrequency domain of said fast Fourier transformed mixed audio signals,wherein said pseudo Doppler shift is a difference between a current peakfrequency versus a closest spectral point below said current peakfrequency in said fast Fourier transformed mixed audio signals; andtranslating said pseudo Doppler shift to an actual Doppler shift toestimate said velocity of said object.
 4. The method as recited in claim3 further comprising: detecting peak frequencies in said fast Fouriertransformed mixed audio signals; filtering peak frequencies at spectralpoints; selecting peak frequencies from remaining unfiltered peakfrequencies; estimating a Doppler shift for each of the selected peakfrequencies; and combining said estimated Doppler shifts to derive saidpseudo Doppler shift.
 5. The method as recited in claim 3 furthercomprising: translating said pseudo Doppler shift to said actual Dopplershift within a time window based on a change in said pseudo Dopplershift within said time window.
 6. The method as recited in claim 3further comprising: selecting one or more peak frequencies in saidfrequency domain of said fast Fourier transformed mixed audio signalsbased on their magnitude and said estimated velocity of said object. 7.The method as recited in claim 1 further comprising: estimating aninitial position of said object using said estimated distance from saidspeaker of said controlled device to said microphone of said controlleddevice via said object; and computing coordinates of said object usingsaid estimated distance from said speaker of said controlled device tosaid microphone of said controlled device via said object, saidestimated velocity of said object and said initial position of saidobject.
 8. The method as recited in claim 7, wherein an initial distanceof said object is estimated by filtering out spectral points, selectinga peak out of the remaining peaks based on their amplitude and/orconsistency with said estimated velocity of said object estimated froman actual Doppler shift, rounding said selected peak to the closestspectral points to form a rounded frequency, and converting said roundedfrequency to said initial distance estimate of said object.
 9. Themethod as recited in claim 1 further comprising: estimating a distancebetween said speaker and said microphone of said controlled device;estimating a distance between speakers of said controlled device usingsaid distance between said speaker and said microphone of saidcontrolled device; and computing coordinates of said object using saidestimated distance from said speaker of said controlled device to saidmicrophone of said controlled device via said object, said estimatedvelocity of said object, said estimated distance between said speakersof said controlled device and an estimated distance between saidspeakers and microphones of said controlled device.
 10. The method asrecited in claim 1 further comprising: detecting an initial gesture; andestimating an initial position of said object in response to detectingsaid initial gesture.
 11. The method as recited in claim 10 furthercomprising: detecting said initial gesture by collecting positive tracesof said fast Fourier transformed mixed audio signals while a user isperforming said initial gesture and collecting negative traces of saidfast Fourier transformed mixed audio signals while said user is notperforming said initial gesture; and matching recently received fastFourier transformed mixed audio signals against said positive ornegative traces using a pattern matching algorithm to detect saidinitial gesture.
 12. The method as recited in claim 1 furthercomprising: filtering out a reflection from static objects.
 13. Themethod as recited in claim 1, wherein said object is a hand or a pen.14. The method as recited in claim 1, wherein said controlled devicecomprises one of the following: a computer, a video game console, anaugmented reality/virtual reality headset, a smartphone, a smart watch,a wearable device, a smart TV and a smart appliance.
 15. A computerprogram product for tracking movement of an object, the computer programproduct comprising a computer readable storage medium having programcode embodied therewith, the program code comprising the programminginstructions for: transmitting audio signals from one or more speakersto said object; receiving samples of said audio signals reflected fromsaid object over a period of time; mixing said received audio signalswith said transmitted audio signals; performing a fast Fourier transformon said mixed audio signals; selecting one or more peak frequencies in afrequency domain of said fast Fourier transformed mixed audio signals;estimating a velocity of said object; and estimating a distance from aspeaker of said controlled device to a microphone of said controlleddevice via said object based on said selected one or more peakfrequencies and velocity of said object.
 16. The computer programproduct as recited in claim 15, wherein the program code furthercomprises the programming instructions for: estimating coordinates ofsaid object using said estimated distance from said speaker of saidcontrolled device to said microphone of said controlled device via saidobject, said estimated velocity of said object, estimated distancesbetween all speakers of said controlled device and an estimated distancebetween each pair of said speakers and microphones of said controlleddevice.
 17. The computer program product as recited in claim 15, whereinthe program code further comprises the programming instructions for:estimating a pseudo Doppler shift in said frequency domain of said fastFourier transformed mixed audio signals, wherein said pseudo Dopplershift is a difference between a current peak frequency versus a closestspectral point below said current peak frequency in said fast Fouriertransformed mixed audio signals; and translating said pseudo Dopplershift to an actual Doppler shift to estimate said velocity of saidobject.
 18. The computer program product as recited in claim 17, whereinthe program code further comprises the programming instructions for:detecting peak frequencies in said fast Fourier transformed mixed audiosignals; filtering peak frequencies at spectral points; selecting peakfrequencies from remaining unfiltered peak frequencies; estimating aDoppler shift for each of the selected peak frequencies; and combiningsaid estimated Doppler shifts to derive said pseudo Doppler shift. 19.The computer program product as recited in claim 17, wherein the programcode further comprises the programming instructions for: translatingsaid pseudo Doppler shift to said actual Doppler shift within a timewindow based on a change in said pseudo Doppler shift within said timewindow.
 20. The computer program product as recited in claim 17, whereinthe program code further comprises the programming instructions for:selecting one or more peak frequencies in said frequency domain of saidfast Fourier transformed mixed audio signals based on their magnitudeand said estimated velocity of said object.
 21. The computer programproduct as recited in claim 15, wherein the program code furthercomprises the programming instructions for: estimating an initialposition of said object using said estimated distance from said speakerof said controlled device to said microphone of said controlled devicevia said object; and computing coordinates of said object using saidestimated distance from said speaker of said controlled device to saidmicrophone of said controlled device via said object, said estimatedvelocity of said object and said initial position of said object. 22.The computer program product as recited in claim 21, wherein an initialdistance of said object is estimated by filtering out spectral points,selecting a peak out of the remaining peaks based on their amplitudeand/or consistency with said estimated velocity of said object estimatedfrom an actual Doppler shift, rounding said selected peak to the closestspectral points to form a rounded frequency, and converting said roundedfrequency to said initial distance estimate of said object.
 23. Thecomputer program product as recited in claim 15, wherein the programcode further comprises the programming instructions for: estimating adistance between said speaker and said microphone of said controlleddevice; estimating a distance between speakers of said controlled deviceusing said distance between said speaker and said microphone of saidcontrolled device; and computing coordinates of said object using saidestimated distance from said speaker of said controlled device to saidmicrophone of said controlled device via said object, said estimatedvelocity of said object, said estimated distance between said speakersof said controlled device and an estimated distance between saidspeakers and microphones of said controlled device.
 24. The computerprogram product as recited in claim 15, wherein the program code furthercomprises the programming instructions for: detecting an initialgesture; and estimating an initial position of said object in responseto detecting said initial gesture.
 25. The computer program product asrecited in claim 24, wherein the program code further comprises theprogramming instructions for: detecting said initial gesture bycollecting positive traces of said fast Fourier transformed mixed audiosignals while a user is performing said initial gesture and collectingnegative traces of said fast Fourier transformed mixed audio signalswhile said user is not performing said initial gesture; and matchingrecently received fast Fourier transformed mixed audio signals againstsaid positive or negative traces using a pattern matching algorithm todetect said initial gesture.
 26. The computer program product as recitedin claim 15, wherein the program code further comprises the programminginstructions for: filtering out a reflection from static objects.
 27. Adevice, comprising: a memory unit for storing a computer program fortracking movement of an object; and a processor coupled to the memoryunit, wherein the processor is configured to execute the programinstructions of the computer program comprising: transmitting audiosignals from one or more speakers to said object; receiving samples ofsaid audio signals reflected from said object over a period of time;mixing said received audio signals with said transmitted audio signals;performing a fast Fourier transform on said mixed audio signals;selecting one or more peak frequencies in a frequency domain of saidfast Fourier transformed mixed audio signals; estimating a velocity ofsaid object; and estimating a distance from a speaker of said controlleddevice to a microphone of said controlled device via said object basedon said selected one or more peak frequencies and velocity of saidobject.
 28. The device as recited in claim 27, wherein the programinstructions of the computer program further comprise: estimatingcoordinates of said object using said estimated distance from saidspeaker of said controlled device to said microphone of said controlleddevice via said object, said estimated velocity of said object,estimated distances between all speakers of said controlled device andan estimated distance between each pair of said speakers and microphonesof said controlled device.
 29. The device as recited in claim 27,wherein the program instructions of the computer program furthercomprise: estimating a pseudo Doppler shift in said frequency domain ofsaid fast Fourier transformed mixed audio signals, wherein said pseudoDoppler shift is a difference between a current peak frequency versus aclosest spectral point below said current peak frequency in said fastFourier transformed mixed audio signals; and translating said pseudoDoppler shift to an actual Doppler shift to estimate said velocity ofsaid object.
 30. The device as recited in claim 29, wherein the programinstructions of the computer program further comprise: detecting peakfrequencies in said fast Fourier transformed mixed audio signals;filtering peak frequencies at spectral points; selecting peakfrequencies from remaining unfiltered peak frequencies; estimating aDoppler shift for each of the selected peak frequencies; and combiningsaid estimated Doppler shifts to derive said pseudo Doppler shift. 31.The device as recited in claim 29, wherein the program instructions ofthe computer program further comprise: translating said pseudo Dopplershift to said actual Doppler shift within a time window based on achange in said pseudo Doppler shift within said time window.
 32. Thedevice as recited in claim 29, wherein the program instructions of thecomputer program further comprise: selecting one or more peakfrequencies in said frequency domain of said fast Fourier transformedmixed audio signals based on their magnitude and said estimated velocityof said object.
 33. The device as recited in claim 27, wherein theprogram instructions of the computer program further comprise:estimating an initial position of said object using said estimateddistance from said speaker of said controlled device to said microphoneof said controlled device via said object; and computing coordinates ofsaid object using said estimated distance from said speaker of saidcontrolled device to said microphone of said controlled device via saidobject, said estimated velocity of said object and said initial positionof said object.
 34. The device as recited in claim 33, wherein aninitial distance of said object is estimated by filtering out spectralpoints, selecting a peak out of the remaining peaks based on theiramplitude and/or consistency with said estimated velocity of said objectestimated from an actual Doppler shift, rounding said selected peak tothe closest spectral points to form a rounded frequency, and convertingsaid rounded frequency to said initial distance estimate of said object.35. The device as recited in claim 27, wherein the program instructionsof the computer program further comprise: estimating a distance betweensaid speaker and said microphone of said controlled device; estimating adistance between speakers of said controlled device using said distancebetween said speaker and said microphone of said controlled device; andcomputing coordinates of said object using said estimated distance fromsaid speaker of said controlled device to said microphone of saidcontrolled device via said object, said estimated velocity of saidobject, said estimated distance between said speakers of said controlleddevice and an estimated distance between said speakers and microphonesof said controlled device.
 36. The device as recited in claim 27,wherein the program instructions of the computer program furthercomprise: detecting an initial gesture; and estimating an initialposition of said object in response to detecting said initial gesture.37. The device as recited in claim 36, wherein the program instructionsof the computer program further comprise: detecting said initial gestureby collecting positive traces of said fast Fourier transformed mixedaudio signals while a user is performing said initial gesture andcollecting negative traces of said fast Fourier transformed mixed audiosignals while said user is not performing said initial gesture; andmatching recently received fast Fourier transformed mixed audio signalsagainst said positive or negative traces using a pattern matchingalgorithm to detect said initial gesture.
 38. The device as recited inclaim 27, wherein the program instructions of the computer programfurther comprise: filtering out a reflection from static objects.